Generative Prior Knowledge for Discriminative Classification
نویسندگان
چکیده
We present a novel framework for integrating prior knowledge into discriminative classifiers. Our framework allows discriminative classifiers such as Support Vector Machines (SVMs) to utilize prior knowledge specified in the generative setting. The dual objective of fitting the data and respecting prior knowledge is formulated as a bilevel program, which is solved (approximately) via iterative application of second-order cone programming. To test our approach, we consider the problem of using WordNet (a semantic database of English language) to improve low-sample classification accuracy of newsgroup categorization. WordNet is viewed as an approximate, but readily available source of background knowledge, and our framework is capable of utilizing it in a flexible way.
منابع مشابه
Exploiting Unlabelled Data for Hybrid Object Classification
We propose a semi-supervised learning algorithm for visual object categorization which utilizes statistical information from unlabelled data to increase classification performance. We build on an earlier hybrid generative-discriminative approach by Holub et al. [6] which extracts Fisher scores from generative models. The hybrid model allows us to combine the modelling power and flexibility of g...
متن کاملSegmenting Glioma in Multi-Modal Images using a Generative-Discriminative Model for Brain Lesion Segmentation
In this paper, we evaluate a generative-discriminative approach for multi-modal tumor segmentation that builds – in its generative part – on a generative statistical model for tumor appearance in multi-dimensional images [1] by using a “latent” tumor class [2, 3], and – in its discriminative part – on a machine learning approach based on a random forest using long-range features that is capable...
متن کاملA Kernel Classifier for Distributions
This paper presents a new algorithm for classifying distributions. The algorithm combines the principle of margin maximization and a kernel trick, applied to distributions. Thus, it combines the discriminative power of support vector machines and the well-developed framework of generative models. It can be applied to a number of real-life tasks which include data represented as distributions. T...
متن کاملDiscriminative Models for Semi-Supervised Natural Language Learning
An interesting question surrounding semisupervised learning for NLP is: should we use discriminative models or generative models? Despite the fact that generative models have been frequently employed in a semi-supervised setting since the early days of the statistical revolution in NLP, we advocate the use of discriminative models. The ability of discriminative models to handle complex, high-di...
متن کاملA Framework for Incorporating Class Priors into Discriminative Classification
Discriminative and generative methods provide two distinct approaches to machine learning classification. One advantage of generative approaches is that they naturally model the prior class distributions. In contrast, discriminative approaches directly model the conditional distribution of class given inputs, so the class priors are only implicitly obtained if the input density is known. In thi...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
- J. Artif. Intell. Res.
دوره 27 شماره
صفحات -
تاریخ انتشار 2006